(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 




lillllllllllllllllllllllllllll 



(43) International Publication Date (10) International Publication Number 

16 October 2003 (16.10.2003) PCT WO 03/085965 Al 



(51) International Patent Classification 7 : H04N 5/445 // 

7/173 

(21) International Application Number: PCT/IB03/01194 

(22) International Filing Date: 2 April 2003 (02.04.2003) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 

0207791.5 



4 April 2002 (04.04.2002) GB 



(71) Applicant (for all designated States except US): KONIN- 
KLIJKE PHILIPS ELECTRONICS N.V. [NL/NL], 
Groenewoudseweg 1, NL-5621 BA Eindhoven (NL). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): STAUNTON, De- 
clan, M. [IE/IE]; c/o Philips Intellectual Property & Stan- 
dards, Cross Oak Lane, Redhill, Surrey RH1 5HA (GB). 
SALOMONS, Eduard, W. [NLflE]; c/o Philips Intellec- 
tual Property & Standards, Cross Oak Lane, Redhill, Sur- 
rey RH1 5HA (GB). 



(74) Agent: TURNER, Richard, C; Philips Intellectual Prop- 
erty & Standards, Cross Oak Lane, Redhill, Surrey RH1 
5HA (GB). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, H, GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NI, NO, NZ, OM, PH, PL, PT, RO, RU, SC, SD, 
SE, SG, SK, SL, TJ, TM, TN, TR, TT, TZ, UA, UG, US, 
UZ, VC, VN, YU, ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European patent (AT, BE, BG, CH, CY, CZ, DE, DK, EE, 
ES, FI, FR, GB, GR, HU, IE, IT, LU, MC, NL, PT, RO, 
SE, SI, SK, TR), OAPI patent (BF, BJ, CF, CG, CI, CM, 
GA, GN, GQ, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— with international search report 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



= (54) Title: DIGITAL REMOTE CONTROL DEVICE 



IT) 
v© 
On 
to 

00 

o 





(57) Abstract: A remote control device 4 is for use with a television receiving apparatus 1. The television receiving apparatus 1 
comprises a system controller 2, a receiver for receiving broadcast video data and supplemental textual data and a TV display 3 for 
displaying the data. The remote control device 4 is operable to communicate with the system controller 2 of the television receiving 
apparatus 1 and has it's own display 8 and user-operable control. The remote control device 4 comprises means responsive to the 
user operable control for requesting and downloading video and/or supplemental textual data received by the television receiving 
apparatus 1. This video and/or supplemental textual data is not currently displayed on the TV display, and is viewed by the user on 
the display 8 of the remote control device 4. 
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DESCRIPTION 

DIGITAL REMOTE CONTROL DEVICE 

5 This invention relates to a remote control for a multimedia TV receiver. In 

particular the invention relates to a remote control with a display and two-way 
communication with a multimedia TV receiver that is configured to enable a 
variety of novel, interactive applications. 

io GB-A-2343073 describes a remote control with a display which is 

configured for the display of electronic programme guides (EPG), teletext and 
internet content remotely from a TV set. The document further describes a facility 
for the instant replay of part of a televised program being watched by a user. 
WO 00/59212 describes a remote control also capable of providing on a 

15 display remote from the TV display, EPG and other information and data. In 
addition, the remote control described in WO 00/59212 is provided with a soft 
graphical user interface (GUI) and speech recognition facility whereby the user ' 
can, for example request information from the EPG related to a specific topic 
(such as sport, movies or a named TV show). 

20 The described prior art remote controllers are limited in function in that 

they provide only information or data specifically requested by the user. Also, 
video images, which are displayable remotely using these prior art devices, are 
restricted by the channel selection being viewed on the TV at the time of use of 
the remote control. 

25 

The present invention aims to further extend the user applications for a 
remote control with a visual display. 

In accordance with a first aspect the present invention provides a remote 
control device for use with a television receiving apparatus, the television 
30 receiving apparatus comprising; a system controller, a receiver for receiving 
broadcast video data and supplemental textual data and a TV display for 
displaying the data, the remote control being operable to communicate with the 
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system controller of the television receiving apparatus and having if s own display 
and user-operable control, wherein, the .remote control device comprises means 
responsive to the user operable-control for requesting and downloading video 
and/or supplemental textual data received by the television receiving apparatus 

5 and not currently displayed on the TV display, to be viewed by the user on the 
display of the remote control device. 

The invention is conveniently implemented where the television receiving 
apparatus system controller comprises two tuners, a first of which is operable to 
find data and information for display on the TV display and a second which is 

io operable to find data and information for display on the display of the remote 
control device. 

In another aspect, the invention provides a television receiving apparatus 
comprising a receiver for receiving broadcast video data and supplemental 
textual data, a TV display for displaying the data and a system controller, the 

15 system controller comprising a plurality of tuners, at least one tuner being 
controllable by means of a user-operable control associated with a remote control 
device in accordance with the first aspect, to display data on the display of the 
remote control device. 

In a further aspect the invention provides a television receiving apparatus 

20 comprising a receiver for receiving broadcast video data and supplemental 
textual data in a plurality of data streams, a TV display for displaying data and a 
system controller, the system controller having means for receiving and 
processing a request from an alternative display device to display data 
associated with a data stream, other than that (or those) displayed on the TV 

25 display, on the alternative display device. 

In another aspect the invention provides a method for displaying images 
and/or information associated with a data stream received by a television 
receiving apparatus on a display device remote from the television receiving 
. apparatus comprising; in response to a request, locating a data stream having 

30 associated therewith the requested images and/or information; 

extracting the located data stream; defining the extracted data stream in a form 
mutually comprehensible to the television receiving apparatus and the remote 
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display device; transmitting the defined data stream to the remote display device; 
and, rendering the images and/or information associated with the defined data 
stream on a display screen of remote display device. 

In yet another aspect, the invention provides A method for providing 
5 interactive participation in a TV programme comprising; in a first data stream, 
broadcasting video of the programme to be displayed on a first display apparatus 
associated with a television receiving apparatus; in a second data stream 
providing information relating to the content of the first data stream to be 
displayed on a second display apparatus, which second display apparatus is 
io provided with a user interface; and, requesting a user response to the information 
contained in the second data stream. 

The invention permits a user to view on the remote control device images 
other than the one being watched on the main TV display. This allows the user to 
follow a program on another channel (for example, to see if an advertisement 
15 break has finished, or a news story of interest has come up) without interfering 
with the viewing of the programme currently on the main TV display. Alternatively, 
the user may view a teletext page associated with a channel other than that 
currently displayed on the TV display, or may access internet web sites or send 
and receive e-mails all without disrupting viewing on the main TV display. 
20 Conveniently, the invention may be enabled by means of an all-software decode 
running on the central processor (CPU) of the set top box (STB) or other system 
controller of an interactive TV receiver. This allows reduced (but satisfactory) 
resolution previews to be displayed on the remote display thereby allowing its 
implementation via relatively simple STB technology with almost zero additional 
25 cost to the manufacturer. 

Desirably, the remote control device is configured to display requested 
data whether or not the TV display is active. When the TV display is active, the 
remote control device will conveniently display images from a TV channel other 
than that being viewed on the TV display. In addition or in the alternative, the 
30 remote control may be configured to request, receive and display images recently 
displayed on the TV display, such as play back of a previous scene in a 
programme currently being viewed on the TV display. This feature may be 
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enabled by interaction between the user operable control and a memory in the 
system controller or by means of a local memory in the remote control device. 

The user operable control may be provided in any convenient form 
including, but not strictly limited to; a keyboard (optionally incorporating a mouse, 
5 tracker ball or similar feature to assist in selecting from on screen menu displays); 
touch screen technology and/or speech recognition. 

The remote control device may communicate with the system controller of 
the TV receiving apparatus by any suitable means. Desirably, communication is 
via a cordless connection such as (but not strictly limited to) infrared. 

io Desirably, the remote control device is configured to interact with other 

audio/visual apparatus such as a DVD player, VCR or games console. In such 
embodiments, the remote control is provided with means responsive to the user 
operable control for requesting, receiving and displaying data or information from 
the alternative audio/visual device other than is currently being displayed on the 

15 TV display. Such features permit a user, for example, to view a programme being 
recorded on a VCR whilst an alternative programme is viewed on the TV display. 
Alternatively, the user may remotely view a TV programme currently being 
broadcast, whilst a movie recorded on a DVD is being played on the TV display. 
It is known for digital TV broadcasts to comprise both video data and 

20 textual information which may comprise separate data streams. The data streams 
may be transmitted in tandem in a single carrier signal. Such arrangements can 
be used to provide interactive data such as the positions of contenders in a race, 
newsflashes, or the latest score in a sports match. It is envisaged that such 
technology could be used to provide an interactive medium via the display of the 

25 remote control device of the invention. For example, the user operable control 
may be configured to download quiz questions to the remote control display and 
to upload answers to these questions. Again, the user may have this interactive 
experience without interfering with the enjoyment of a TV programme by other 
viewers of the main TV display. 

30 In the case of a suitably live broadcast sporting event, position data of 

players/participants may transmitted by a broadcaster in tandem with a live 
broadcast (for example, by image processing on a video feed from a fixed 
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camera). In addition, a graphic representation of the players may be 
superimposed on a representation of the course/pitch/track at which the live 
sporting event is occurring and downloaded for display on the screen of the 
remote control device whilst the live action is broadcast on the main TV display. A 
continuously updated field displaying event statistics may augment the graphic 
representation. By interaction with the remote control device by means of the 
user-operable display, the user may choose control options for the display of the 
event on the TV display. For example, the user may select camera angles, audio 
feed locations and the like simply by indicating desired locations on the graphic 
representation. The user selections may then be uploaded to the broadcaster via 
the system controller of the TV receiving apparatus and the camera view/audio 
feed may be transmitted to the remote and/or TV display in accordance with the 
user's requirement. 

In addition, or in an alternative, where the live action video and graphic 
representation are stored in conjunction, the graphic representation (stepped 
through at high speed) would allow a user to quickly identify interesting passages 
of play and then mark them to create a highlights package from the recorded 
video. 

In a further development of the aforementioned interactive applications, it 
is envisaged a broadcasting company could encourage channel loyalty through a 
"loyalty points" scheme. The "points" are provided in the form of electronic credits. 
Such schemes are well known in the retail industry where purchasers are 
awarded points electronically via a loyalty card in return for spending a certain 
amount of money in a store. In the proposed scheme of the invention, 
broadcasters send points (carried in an electromagnetic signal) to a TV receiver 
embedded in a video broadcast. The remote control device is provided with a 
user operable control for requesting and downloading these loyalty points and a 
memory for storing the points. In further broadcasts, the broadcaster may send 
interactive information directly to the remote control device via the same route as 
the points. Such interactive information may provide the user with opportunities to 
spend loyalty points or to acquire additional points, for example by betting points 
on the outcome of a future event posed by the broadcast interactive information. 
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Points accrued may be used to purchase items or services offered for sale 
by the originating broadcaster. For example, points may be traded for viewing of 
a pay-per-view presentation. Alternatively, character merchandise associated with 
a flagship TV show broadcast by the broadcaster may be purchased with the 
5 loyalty points in place of actual currency. In a further alternative, telephone calls 
or Internet access made utilising the same service provider as supplies the 
interactive TV data may be paid using accrued points. Points may be awarded by 
the broadcaster as a reward for certain behaviours, for example watching 
advertisements or buying goods or services. 
lo A particular embodiment provides a gaming/gambling arrangement based 

on the "loyalty points" system, and having a number of particular features: points 
may be given freely by a broadcaster to encourage the user to gamble with them. 

The gaming/gambling activity is in the form of questions appearing on the 
remote display, relating to a TV programme which the user is currently watching 
i 5 on the main TV display. The remote control device is suitably configured such 
that the broadcaster can "wake up" the device from a dormant state to present 
questions to the user; and user response (or lack of it) to gaming questions may 
be used to build up a user profile. User profiles may subsequently be used by the 
broadcaster for selective advertising and marketing purposes. For example, 
20 where a user is recognised to respond to questions related to sporting 
presentations, the broadcaster may target the user with advertisements directed 
to sporting magazines, sports clothes and future, pay-per-view sporting 
presentations to be broadcast by the broadcaster. 

Other possibilities will no doubt occur to the skilled artisan without 
25 departing from the true scope of the invention. 

In another aspect, the remote control device of the invention may be used 
in association with digital recording apparatus, for example a video 
camera/recorder. Scenes recorded by the camera/recorder could be remotely 
downloaded and edited using the remote display and an appropriately configured 
30 user interface. The complete edited version could then be uploaded back to the 
camera and recorded on a data carrier for reproduction and personal use. 

In a further interactive application, a broadcaster may have the facility to 
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"wake" the remote control device from a dormant state, for example by 
downloading an activation code to the device or the system controller. Once the 
device has been awakened, the broadcaster may selectively send data and 
information to the remote control device for display. Information maybe provided, 
5 for example, in the form of downloads of virtual leaflets containing advertising 
data related to material that the user is currently viewing on the TV display. 
Where the programme being viewed is a commercial, the data may comprise 
background information or details of other items in a similar product range. If the 
user chooses to ignore the downloaded leaflet, it is removed automatically from 
io the remote control device after a defined period, however, if the user 
acknowledges receipt of the leaflet within the defined period, by interaction with 
the remote control device, details on the subject matter of the leaflet and the 
nature of the user's response is passed back to the broadcaster who may use the 
data to build up a personal profile of the viewer. The profile may be used to 
i 5 selectively target individuals with advertisements for goods or services in which 
they have historically shown an interest. Such advertising may include the 
promotion of TV broadcasts on other channels (provided by the same 
broadcaster) which may be of interest to the user. Leaflets may be sent 
repeatedly and, where the local system permits, they may be locally cached to 
20 avoid the download of such large volumes of data. 

Embodiments of the invention and various aspects thereof will now be 
described. For the purposes of this description, the following terms are to be 
interpreted as defined. 

"Broadband" A transmission medium capable of supporting a wide range 
25 of frequencies. In the context of the described invention, this refers to a high- 
speed (typically 51 2 kbit/s but up to 40 Mbit/s is possible) Internet connection via 
cable, DSL, satelite or wireless local loop. 

"Gateway" A device that contains a broadband modem, a wireless 
interface and the associated software to make it an intelligent hub. This may be 
3 o a set top box (STB), a personal computer (PC) or a stand-alone device. 

The following acronyms are believed to be commonly used and 
understood in the technical field of the invention: 
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ADSL Asymmetric Digital Subscriber Line 

DCT Discrete Cosine Transformation 

DSL Digital Subscriber Line 

DVB Digital Video Broadcast 

EPG Electronic Program Guide 

FLTK Fast Light Tool Kit 

GUI Graphical User Interface 

I DCT Inverse Discrete Cosine Transformation 

MPEG Motion Picture Experts Group 

PDA Personal Digital Assistant 

PES Packetised Elementary Stream 

POTS Plain Old Telephone System 

PSI Program Specific Information 

SI Service Information 

STB Set-Top Box 

VLC Variable Length Coding 

XML Extensible Mark-up Language 

Desirably, the invention is in the form of a communicating portable touch 
screen display device that allows information normally displayed on the TV 
screen to be displayed 'off-screen' and on the display of the remote, touch screen 
device. Information may be displayed on the device that is not currently 
displayed on the TV screen, for example a preview image of another channel or a 
web page associated with a program being watched. This remote display 
capability allows a user to find out whatever information he/she likes, such as 
browsing EPG listings or viewing teletext pages on whatever channel, without 
disturbing the viewing of persons watching the main TV display. Also, when 
appropriately configured, the device enables the user to experience a wide range 
of interactive activities. 

The content displayed by the remote control device of the invention is 
accessed through a gateway device such as a STB, and the device has no direct 
external network connection of its own. It is envisaged that a plurality of such 
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devices may be used simultaneously, yet independently, in relation to a single TV 
receiver. For example, each member in a family may have their own personal 
device. In an alternative, a single device may be equipped with a memory facility 
whereby a plurality of individual user set ups may be stored and selected by 

5 individual users. 

Whilst the description of the invention focuses on the device as a TV 
companion, it should be appreciated that the device has potential application as a 
universal front end for digital consumer devices such as PVRs, DVDs and MP3 
jukeboxes. As these digital consumer devices offer more and more functionality, 

io their user interfaces will become increasingly complex and the device of the 
invention is a highly appropriate platform to present such user interfaces. 

There are two principal features of the device; firstly, the ability to mine 
information in the gateway device, and secondly, the facility to display images or 
data on the remote display which relate to different transmissions than that or 

15 those displayed on the main TV display. 

For the purposes of exemplification, an embodiment of the invention will 
now be further described with reference to the Figures in which: - 

Figure 1 illustrates the system architecture for an embodiment of a remote 
20 display device in accordance with the invention, and 

Figure 2 illustrates the software architecture for an embodiment of a 
remote display device in accordance with the invention. 

Figure 1 illustrates how content arrives at the remote control device when it 
25 is used as a digital TV companion device: 

In Figure 1 , a television receiving apparatus 1 is situated in a user's home 
environment, H. The television receiving apparatus 1 comprises a system 
controller embodied in a STB 2, and a display, 3. In wireless communication with 
the apparatus 1 and also in the home environment H is a remote control device 4 
3 o as previously described. In a location remote from H, is located the sen/ice 
provider (for example a TV broadcaster) SP, who transmits his own generated 
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content 5 to receiving apparatus 1 in environment H. In addition, service provider 
SP may also receive, for example, Internet data 6 and content from partner 
websites 7. The content 6 and 7 can be requested by the television receiving 
apparatus 1 and transmitted via the service provider SP to the STB 2. As 
5 previously described, the remote control device 4 can in turn instruct the STB 2 to 
request certain content 5, 6, 7 from the service provider SP for display on the 
display 8 of the remote control device 4. 

As shown in the Figure, the link between the service provider SP and the 
home (or location of the user) H may be any of a satellite, cable, terrestrial 
o broadcast or a telecommunications operator or any other conduit for providing 
digital TV data. The link is composed of several multiplexes, each of which may 
carry several video, audio and/or data streams. The skilled addressee will be 
aware of various methods by which the various streams are multiplexed together 
as these have been previously described in relation to DVB technology. Several 
15 different types of data streams are specified, most notably SI (such as teletext 
EPG or Internet) and PS1 (the video/audio broadcast). 

As is also known, certain DVB specifications allow any type of digital 
information to be transmitted as a private data stream. This data stream may be 
sent to all receivers, individual receivers or selected groups of receivers and by 
ao means of suitable software, the device of the invention can be configured to 
receive this data independently of the data for display on the TV screen. The 
WAN connection shown in Figure 1 is optional, it allows downstream internet data 
to be transmitted to the device of the invention (the upstream connection is not 
shown in the Figure, but could, for example, be a POTS or ADSL modem, a cable 
25 modem or even a satellite back channel). This arrangement facilitates accessing 
of both Internet images and TV video broadcasts simultaneously (an activity 
sometimes referred to as "telewebbing"). Enhanced service (EPG, teletext and 
the like) delivery is also made possible by this arrangement. 

The following describes the method by which information is caught and 
3 o displayed by the device of the invention. 

The display of information on an external device involves a number of 
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stages. The information must first be discovered, then extracted, rendered, 
transmitted and displayed. 

Discovery : When an item of information (for example, an EPG description 
or an l-frame for a preview image) is requested, the first step is to determine 
5 where that information resides. It may be part of an already existing (for example 
in the STB memory) SI table or it may exist in a different transport stream than 
the one currently being examined. The discovery phase resolves the location of 
the item of information being requested. 

Extraction : Once the location of the information item has been determined 

io the appropriate information is extracted. The mechanism for extraction will 
depend on the relevant location. If the item was not available in either the STB 
memory or the data stream for the channel currently selected for the TV display, 
then the tuner is instructed to change to the correct frequency (as determined 
during the discovery phase) for receiving the requested information. The PID and 

15 section filters may be configured to build a table that may be examined for the 
requested information item. 

Rendering: With traditional on-screen display of information the rendering 
phase would consist of drawing the appropriate images on the graphics planes 
which are superimposed on the TV picture. The inventive device provides for the 

20 rendering of images on a display other than the TV screen (i.e. the remote 
display). The rendering stage therefore, consists of describing the item of 
information in a form mutually comprehensible to both the gateway device and 
the remote control device of the invention. In the described embodiment, this is 
achieved using XML and the appropriate DTD/schema. 

25 Transmission : The XML description of the requested information is 

transmitted wirelessly to the remote device. This can be achieved by using any 
number of existing wireless protocols known in the technical field of the invention. 
For example, Bluetooth, IEEE 802.1 1 or HomeRF. 

Display : Once the XML description of the requested information item has 

30 been successfully received it is rendered on the display of the remote control 
device. 
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The following describes the functionality of an embodiment of the invention 
for some user applications. Examples of suitable hardware and software 
configurations for an embodiment of the invention are now outlined. 

Two suitable platforms are the Intel StrongARM SA1 1 1x reference board 
and the Compaq iPAQ H3630 PDA. The iPAQ device features very similar 
hardware (in particular, the same CPU) to the StrongARM reference board and 
provides a subset of the StrongARM functionality in a more compact form factor. 

In the embodiment described, the STB is a low-end box based on the 
SGS-Thomson STi5512 chip. This chip features a 60 MHZ ST20 32-bit 
processor that provides sufficient processing power to control the STB and run all 
native applications. There is a limited amount of processing power (the exact 
amount depends on what else needs to be done simultaneously) available to run 
other applications such as the l-frame grabber or EPG miner incorporated in the 
present invention. More advanced STBs feature higher performance processors 
and it is envisaged that the increased processing power provided by these 
advanced STBs may enable a wider range of applications to be implemented in 
the device of the invention and delays in providing images on the remote screen 
to be decreased. Desirably, the STB (or other gateway device) is provided with a 
second tuner which is separate from that which is used to select a channel for 
display on the TV display, the second tuner being controllable by the remote 
control device of the invention. STBs are known to be available which feature a 
second tuner for the purpose of recording a program other than the one being 
watched to a hard disk and it is expected that use could be made of this existing 
technology to enable many of the applications envisaged for the present 
invention. When a second tuner is present it is possible to access information 
(data or pictures) from any transport stream received by the TV receiver. 

The device of the invention may be connected to the STB (or other 
gateway device) by a serial cable but is preferably connected by a wireless link, 
such as a Bluetooth link. This can be achieved by attaching Bluetooth 'dongle' 
modules to the serial ports of the gateway devicie and the remote control device. 
It is envisaged that existing, installed STBs and other gateway devices may be 
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upgraded in the field to provide functionality for the remote control device of the 
invention, by attaching such dongle modules and upgrading the flash memory of 
the gateway device with the relevant code. This provides a significant cost 
advantage to suppliers of gateway devices already in the field, who may provide 
a range of novel sen/ices to the consumer by means of a simple, low cost add-on 
to the existing generation of STBs and other gateway devices. 

An example of a suitable software framework for the invention is shown in 
Figure 2 below. The system requires a protocol for the transfer of commands 
and data between the remote control device and the gateway device (STB). 
The software stack used on the remote control device is based predominantly on 
freely available open source code. The software implements novel protocols to 
exchange information, interpret information and display the information using an 
appropriate GUI, for the device of the invention. 

The system enables decoding of a second video stream in addition to a 
first that is decoded by the STB for display on the TV. The decoding of the 
second stream is conveniently carried out by means of suitably configured 
software on the main processor of a gateway device such as an STB. So as not 
to conflict with tasks already loaded on the processor and required by the STB or 
the cycle budget for these tasks (which can be unpredictable), the software 
decode desirably has a low processing load and is scalable so that it still works to 
satisfaction even at times when the available cycle budget of the STB processor 
is reduced. 

In a preferred embodiment, both temporal (throwing away frames) and 
spatial (reducing the resolution) scalability are used. In one example, only I- 
f rames are decoded and resolution is reduced eight times both horizontally and 
vertically. This can be implemented using the following method 

i) A PES filter for the 2nd data stream is set up in the demux of the 
decoder 

ii) Data is written into the PES buffer by the hardware demux 
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iii) If a full l-frame is found in the buffer, then decoding starts. (The 
decoding process is described in more detail below) 

iv) The decoded image is then sent to the display (or GUI) of the 
remote control device. Before display, the image format of the l-frames may be 

5 mapped onto a palette 

v) The PES filter is re-enabled and the cycle repeats. 

In one embodiment, l-frames are decoded and only the DC components of 
the 8x8 blocks are calculated. This gives a reduction in resolution by x 8. With 
this reduced resolution it is possible still to detect changes in image content (for 
io example, whether a commercial break has ended and a soccer match resumed). 
A significant amount of processing load may be saved by using only DC 
coefficients, since this eliminates the need for performing IDCT operations. All 
DCT coefficients 8x8 blocks, (so also DCs) are coded in the incoming MPEG2 bit 
stream as VLC words. The DC coefficient is located at the start of each macro 
i 5 block. However macro blocks are not byte aligned. This means that in order to 
get to the DC of the next macro block, all AC coefficients of the previous macro 
block must be skipped. This implies that basically a full variable length decode 
operation must be performed, just like for full resolution decoding. This is costly 
bit-wise processing and consumes by far the largest part of the processing load. 
20 As only the length (rather than the exact value) of the AC coefficients are 

required, to skip AC coefficients, it is possible to process the VLC coefficients 
more efficiently by using characteristics of the MPEG VLC tables. For example, 
analysis of bits for a particular coefficient is ceased once the length of the 
coefficient is known, and the remaining bits can be skipped. The desired result of 
25 the decoding is to end up with a progressive picture. This means that for macro 
blocks or DCT blocks that are field coded, field to frame conversion must be 
performed. This is can be achieved simply by averaging the DC components of 
the fields. 

The channel to be viewed in the display of the remote device may be 
3 o changed without interference with the image on the main TV display. When 
requested by the user the remote control device sends a channel change 
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command to the STB. The XML interpreter application handles this command by 
passing the channel number to the channel change function of the STB 
middleware. Many PDAs feature a built-in IR port which could also be used to 
control television functions such as volume, brightness etc if the correct remote 

5 control codes were known to the PDA. 

Other potential applications of the device of the invention not described in 
detail here include: remote teletext display of information from channels other 
than that viewed on the TV display; localisation of advertising and content; 
interactive and 'push' advertising; remote monitoring of programs viewed on the 

io main TV display; immersive applications for sports programs; consumer loyalty 
schemes; universal front end/remote control; new user interfaces; EPG search 
engines; instant messaging on the remote display; alternate views for gaming 
consoles. 

The remote device 4 can receive information from a user via standard user 
15 interfaces such as keypad, touch screen including handwriting recognition and 
voice control. 

It is to be understood that the foregoing is merely exemplary of some 
embodiments and applications of the invention and is not intended to be limiting 
on the scope of the invention as defined in the appended claims. 



20 
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CLAIMS 

1. A remote control device for use with a television receiving 
apparatus, the television receiving apparatus comprising; a system controller, a 

5 receiver for receiving broadcast video data and supplemental textual data and a 
TV display for displaying the data, the remote control device being operable to 
communicate with the system controller of the television receiving apparatus and 
having it's own display and user-operable control, wherein, the remote control 
device comprises means responsive to the user operable control for requesting 

io and downloading video and/or supplemental textual data received by the 
television receiving apparatus and not currently displayed on the TV display, to 
be viewed by the user on the display of the remote control device. 

2. A remote control device as claimed in claim 1 wherein the system 
15 controller of the television receiving apparatus comprises two or more tuners one 

of which is controllable by means of the user-operable control to find data and 
information for display on the display of the remote control device. 

3. A remote control device as claimed in claim 1 or claim 2 wherein the 
20 user operable control comprises a user interface which includes an alphanumeric 

keyboard 

4. A remote control device as claimed in claim 3 wherein the keyboard 
is a virtual keyboard displayed in the display of the remote control device and the 

25 display incorporates touch screen technology. 

5. A remote control device as claimed in any of claims 1 to 3 wherein 
the user operable control comprises a user interface which includes touch screen 
technology incorporated into the display of the device. 

30 

6. A remote control device as claimed in any preceding claim wherein 
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the user operable control comprises a user interface which includes speech 
recognition. 

7. A remote control device as claimed in any preceding claim wherein 
5 the remote control device is operable to communicate with the system controller 

by cordless means. 

8. A remote control device as claimed in claim 7 wherein the cordless 
means are infra red. 

io 

9. A remote control device as claimed in claim 7 wherein the cordless 
means are Bluetooth, IEEE 802.1 1 or HomeRF. 

10. A remote control device as claimed in any preceding claim further 
15 comprising a memory for storing data and information received by the device. 

11. A remote control device as claimed in any preceding claim wherein 
the remote control is provided with means responsive to the user operable control 
for requesting, receiving and displaying data or information from an alternative 

20 audio visual device other than is currently being displayed on the TV display. 

12. A remote control device as claimed in claim 11 wherein the 
alternative audio-visual device is selected from a VCR, a video camera, a DVD 
player and a games console. 

25 

13. A television receiving apparatus comprising a receiver for receiving 
broadcast video data and supplemental textual data in a plurality of data streams, 
a TV display for displaying data and a system controller, the system controller 
having means for receiving and processing a request from an alternative display 

30 device to display data associated with a data stream other than that (or those) 
displayed on the TV display on the alternative display device. 
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14. A television receiving apparatus comprising a receiver for receiving 
broadcast video data and supplemental textual data, a TV display for displaying 
the data and a system controller, the system controller comprising a plurality of 

5 tuners at least one tuner being controllable by means of the user-operable control 
associated with a remote control device as claimed in any preceding claim to 
display data on the display of the remote control device. 

1 5. A method for providing interactive participation in a TV programme 
io comprising; 

in a first data stream, broadcasting video of the programme to be displayed on a 
first display apparatus associated with a television receiving apparatus; 
in a second data stream providing information relating to the content of the first 
data stream to be displayed on a second display apparatus, which second display 
15 apparatus is provided with a user interface; and 

requesting a user response to the information contained in the second data 
stream. 



16. A method as claimed in claim 15 further comprising; 

20 receiving a user response and updating and re-transmitting the information in the 
second data stream based on the nature of the user response. 

17. A method as claimed in claim 15 or 16 wherein the first display 
apparatus is a TV display and the second display apparatus is a display provided 

25 on a remote control device. 

18. A method as claimed in any of claims 1 5 to 1 7 wherein the second 
data stream comprises questions about the content of the programme being 
broadcast via the first data stream. 

30 

19. A method as claimed in claim 18 wherein the requested user 
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response is an answer to the questions in the second data stream. 

20. A method as claimed in any of claims 1 5 to 1 9 wherein the second 
data stream comprises electronic credits for use by the recipient. 

5 

21. A method as claimed in any of claims 18 to 20 wherein the 
questions are presented with associated odds and the user response includes an 
acceptance of the odds and authorisation to debit a defined number of credits 
electronically from an account held by the user. 

io 

22. A method as claimed in claim 21 wherein the updated second data 
stream responsive to the user response includes credits amounting to a value 
based on a value of credits authorised for debit in the user response, the odds 
presented with the questions and the correctness of the user response. 

15 

23. A method as claimed in claim 21 or 22 wherein the first data stream 
comprises video of a real time, live action event and the questions relate to 
actions anticipated but not yet carried out in the live action event. 

20 24. A method as claimed in claim 23 wherein the live action event is a 

sporting event. 

25. A method as claimed in any of claims 21 to 24 wherein the credits 
correspond to actual currency in a users personal account. 

26. A method as claimed in any of claims 21 to 24 wherein the credits 
correspond to electronic credits of virtual currency originated by the broadcaster 
of the programme and provided to the user through previous interactions. 

3 o 27. A method as claimed in any of claims 15 to 17 wherein the 

information in the second data stream comprises an invitation to purchase goods 
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or services and the user response comprises an acceptance of the invitation and 
authorisation to debit a defined number of credits electronically from an account 
held by the user. 

5 28. A method as claimed in any of claims 1 5 to 27 further comprising 

recording the user response whereby to build up a profile of the user, and 
selectively targeting the user with information which the profile suggests is of 
particular interest to the user. 

o 29. A method as claimed in claims 15 to 17 wherein the first data 

stream comprises video of a real time, live action event; the second data stream 
includes options for perspectives from which the event may be viewed by the 
user, and the updated second data stream comprises a view of the event from 
the perspective selected by the user in his response. 

30. A method as claimed in claim 29 wherein the first data stream is 
also updated in response to the user response thereby to provide the selected 
perspective view of the live action broadcast on the first display apparatus. 

20 31 . A method for displaying images and/or information associated with 

a data stream received by a television receiving apparatus on a display device 
remote from the television receiving apparatus comprising; 
in response to a request, locating a data stream having associated therewith the 
requested images and/or information; 

25 extracting the located data stream; 

defining the extracted data stream in a form mutually comprehensible to the 
television receiving apparatus and the remote display device; 
transmitting the defined data stream to the remote display device; and 
rendering the images and/or information associated with the defined data stream 

30 on a display screen of remote display device. 
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32. A method as claimed in claim 31 wherein the step of extracting 
involves instructing a tuner in the TV receiving apparatus to change to a specified 
frequency. 

5 33. A method as claimed in claim 31 or 32 wherein the step of defining 

involves the use of XML. 

34. A method as claimed in any of claims 30 to 33 wherein the step of 
transmitting involves the use of a wireless protocol. 

io 

35. A method as claimed in claim 34 wherein the wireless protocol is a 
Bluetooth protocol. 



WO 03/085965 



PCT/IB03/01194 



1/2 




WO 03/085965 



PCT/IB03/01194 




INTERNATIONAL SEARCH REPORT 



Inter lal Application No 

PCl/lB 03/01194 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (dassification system followed by classification symbols) 

IPC 7 H04N 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the International search (name of data base and, where 

EPO- Internal, WPI Data 



practical, search terms used) 



C« DOCUME 

Category 0 


NT3 CONSIDERED TO Be HEIEVWM 1 ___ 

Citation of document, with indication, where appropriate, of the relevant passages : 


Relevant to claim No. 


X 


US 6 097 441 A (ALLPORT DAVID E) 

1 August 2000 (2000-08-01) 

column 3, line 48 -column 5, line 40 

abstract 


1-35 


A 


US 2002/002707 Al (DO TUAN Q ET AL) 
3 January 2002 (2002-01-03) 
the whole document 


1-35 


P,X 


WO 03 017650 A (DIGE0 INC) 
27 February 2003 (2003-02-27) 
page 6, line 17 - line 18 
abstract; figures 1A,1B 


1-35 


P.A 


EP 1 276 318 A (TV LAB SPA) 
15 January 2003 (2003-01-15) 
the whole document 


1-35 



□ 



Further documents are listed In the continuation of box C. 



m 



Patent family members are Isted In annex. 



° Special categories of cited documents : 

"A" document defining the genera) state of the art which Is not 

considered to be of particular relevance 
"E" earlier document but published on or after the international 

filing date 

*L* document which may throw doubts on priority dajm(s) or 
which is cited to establish the publicafion date of another 
citation or other special reason (as speeffled) 

-0° document referring to an oral disclosure, use. exhibition or 
other means 

■p» document published prior to the international fiSng date but 
later than the priority date claimed 



T later document published after the international filing date 
or priority date and not in conffict with the application but 
cited to understand the principle or theory underlying the 
Invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
Involve an inventive step when the document is taken alone 

"V* document of particular relevance; the claimed Invention 

cannot be considered to involve an inventive step when the 
document Is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the ait 

document member of the same patent family 



Date of the actual completion of the international search 

16 June 2003 



Date of mailing of the international search report 

24 « 



Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patenuaan 2 
NL - 2280 HV Rjswijk 
Tel. (+31-70) 340-2040. Tx. 31 651 epo nl, 
Fax: f*31 -70) 340X3016 



Authorized officer 



HENRI K ANDERSSON/JA A 



Rim PCT/BAS10 feetend sheet) (July 1992) 



INTERNATIONAL SEARCH REPORT 



tal Application No 

PC1/1B 93/01194 



Patent document 
cited in search report 


Publication 
date 


Patent family 
members) 


Publication 
date 


US 6097441 


A 


01-08-2000 


AU 


2010599 A 


19-07-1999 






CA 


2315619 Al 


08-07-1999 








EP 


1044564 Al 


18-10-2000 








JP 


2002502138 T 


22-01-2002 








wo 


yyj4Dyy ax 


08-07-1999 








us 


6567984 Bl 


20-05-2003 


US 20G2O027G7 


Al 


03-01-2002 


NONE 






WO 03017650 


A 


27-02-2003 


US 


2003035075 Al 


20-02-2003 






WO 


03017650 Al 


27-02-2003 


EP 1276318 


A 


15-01-2003 


EP 
EP 


1276319 Al 
1276318 Al 


15-01-2003 
15-01-2003 



Fcrm PCT/ISW210 (patent tamOy annex) UHy 1992) 



